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1 Introduction 



Socio-economic systems can be viewed as systems of many interacting agents or players (see 
for example Santa Fe collection of papers on economic complex systems [1] and econophysics 
papers on Minority Game [2]). We may then try to derive their global behavior from individual 
interactions between their basic entities. Such approach is fundamental in statistical physics 
which deals with systems of many interacting particles. We will explore similarities and dif- 
ferences between systems of many interacting players maximizing their individual payoffs and 
particles minimizing their interaction energy. 

We will consider here game-theoretic models of many interacting agents [3, 4]. In such 
models, agents have at their disposal certain strategies and their payoffs in a game depend on 
strategies chosen both by them and by their opponents. In spatial games, agents are located on 
vertices of certain graphs and they interact only with their neighbors [5, 6, 7, 8, 9, 10, 11, 12]. 
The central concept in game theory is that of a Nash equilibrium. A configuration of strategies 
(an assignment of strategies to agents) is a Nash equilibrium, if no agent, for fixed strategies of 
his opponents, can increase his payoff by deviating from his current strategy. In spatial models, a 
Nash equilibrium is called a Nash configuration. We see that the notion of a Nash configuration 
is similar to that of a ground-state configuration in systems of interacting particles. 

In most models with many players, their strategic interaction is decomposed into a sum of 
two-player games. Only recently there have appeared some systematic studies of truly multi- 
player games [13, 14, 15]. Here we consider spatial games with players located on vertices of 
the triangular lattice. Each agent plays six three-player games with his neighbors on the same 
triangle. 

One of the fundamental problems in game theory is that of the equilibrium selection in games 
with multiple Nash equilibria. We will discuss here the dynamic approach to that problem. It 
may happen that only some equilibria are asymptotically stable in some specific dynamics. We 
will be concerned here with a particular stochastic dynamics. Namely, at discrete moments of 
time, a randomly chosen player may change his strategy. He adopts with a high probability 
a strategy which is the best response to strategies of his neighbors, that is a strategy that 
maximizes the sum of the payoffs of individual games, and with a small probability, representing 
the noise of the system, he makes a "mistake" . Such process is repeated infinitely many times. 
To describe the long-run behavior of stochastic dynamics, Foster and Young [16] introduced 
a concept of stochastic stabihty. A configuration of strategies is stochastically stable if it 
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has a positive probability in the stationary state of the above dynamics in the zero-noise hmit, 
that is the zero probabihty of mistakes. It means that in the long run we observe it with a 
positive frequency. However, for any arbitrarily low but fixed noise, if the number of players is 
big enough, the probability of any individual configuration is practically zero. It means that 
for a large number of players, to observe a stochastically stable configuration we must assume 
that players make mistakes with extremely small probabilities. However, as indicated by van 
Damme and WeibuU [17], a small probability of mistakes should involve some some cost of 
learning strategies played by neighbors. To avoid paying these prohibitively big costs, players 
settle for regimes with low but not extremely low noise. On the other hand, it may happen that 
in the long run, for a low but fixed noise and sufficiently big number of players, the stationary 
state is highly concentrated on an ensemble consisting of one Nash configuration and its small 
perturbations, i.e. configurations, where most players play the same strategy. We will call such 
configurations low-noise ensemble stable. 

We will investigate here the effect of the noise level and the number of players on their long- 
run behavior. In the first part of our paper we will consider the so-called potential games [18]. 
In such games, if any single player changes his strategy, then the payofi^ differences arc the same 
for all players. This is in absolute analogy to systems of interacting particles, where instead of 
maximizing payoffs, particles minimize their interaction energy. We will exploit this analogy 
to describe long-run behavior of potential three-player games with two Nash configurations. 
We will show that a configuration can be stochastically stable but nevertheless may appear in 
the long run with an arbitrarily small frequency if the number of players is large enough - it 
is not low-noise ensemble stable. In the second part of our paper we will present an example 
of a simple nonpotential three-player spatial game, where stochastic stability depends on the 
number of players. 

In Section 2, we introduce spatial three-player games. In Section 3, we compare stochas- 
tic and ensemble stability in potential games. In Section 4, we discuss nonpotential games. 
Discussion follows in Section 5. 

2 Spatial three-player games 

Let A be a finite subset of the triangular lattice. Every site of A is occupied by one player 
who has at his disposal one of two different strategies. Let S — {A, B} be the set of strategies, 
then Qa = is the space of all possible configurations of players. For every i e A, is 



3 



the strategy of the i— th player in the configuration X e and denotes strategies of all 
remaining players; X therefore can be represented as the pair (Xj, U : S x S x S ^ R 

is a payoff function of our game. Without loss of generahty (see a discussion below) it can be 
represented by two matrices: 

where the ij entry, i, j — A, B, of the first matrix is the payoff of the first (row) player when he 
plays strategy i, the second (column) player plays the strategy j and the third (matrix) player 
plays the strategy A; the second matrix represents payoffs of the first player when the third 
player plays the strategy B. We assume that all players are the same and hence payoffs of a 
column and a matrix player can be easily deduced from the above matrices; such games are 
called symmetric. 

Every player interacts only with his nearest neighbors and his payoff is the sum of the 
payoffs resulting from individual games (six games on the triangular lattice). We assume that 
he has to use the same strategy for all neighbors. For X e we denote by Ui{X) the payoff 
of the i—th player in the configuration X: 

iy,{X) = Y.U{Xi,Xj,X,), (2) 

where the summation is with respect to six elementary triangles containing i. 

Definition 1 X e is a Nasii configuration if for every i G A and e S, i/i(A'j,X_j) > 
i^i{Yi,X_,). 

Let us note that the notion of a Nash configuration involves not only payoff functions but 
also the spatial structure of players. It is similar to the notion of a ground-state configuration 
in classical lattice-gas models of interacting particles. However, there are differences. One 
cannot decrease the energy of a ground state-configuration by any local change of particles. 
Prom this follows the existence of a ground-state configuration for any model with finite-range 
interactions. In the definition of a Nash configuration we are allowed to make only one-site 
changes. As a consequence of this restriction, a Nash configuration may not exist. We will be 
not concerned here with such situations. 

Let us notice that if a > and c > 0, then there are two homogeneous Nash configurations: 
X"^ and X^, where all players play the same strategy, A or B respectively. If a > 0, c < and 
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6 > 0, then we have a homogeneous Nash configuration X"^ and three configurations, related 
by translations, where on every elementary triangle there are two B players and one A player. 
We donote by X"^^^ one of these configurations. We see that for above payoff parameters, 
there are multiple Nash configurations. We are therefore faced with a standard game-theoretic 
problem of equilibrium selection. In the following, we will discuss one of the dynamics used in 
evolutionary game theory. 

We start with the deterministic dynamics of the best-response rule. Namely, at each 
discrete moment of time t = 1,2, a randomly chosen player may update his strategy. He 
simply adopts the strategy, Xj, which gives him the maximal total payoff lyiiXj, X^^^) for given 
Xti^, a configuration of strategies of remaining players at time t — 1. 

Now we allow players to make mistakes with a small probability, that is to say they may 
not choose the best response. It is reasonably to expect that the probability of making an error 
should increase if payoffs from alternate strategies approach the payoff of the best-response 
strategy. We will consider here a well-known in statistical mechanics exponential rule which is 
used in game-theoretic and economic literature under the name of the log-linear rule [5, 19]. 
Wc assume that the probability of chosing by the i—th player the strategy Xj at time t is given 
by the following conditional probability: 

where 1/(3 > measures the noise level. 

Let us observe that if /? oo, pf converges pointwise to the best- response rule. Such 
stochastic dynamics is an example of an irreducible Markov chain with l^"^] states (there is a 
nonzero probability of transition from any state to any other state in finite number of steps). 
Therefore, it has the unique stationary probability distribution (also called a stationary state) 
denoted by //^. The following definition was introduced by Foster and Young [16]: 

Definition 2 X e 0,a is stochastically stable i/lim^_>oo A*a(-'^) > 0. 

If X is stochastically stable, then the frequency of visiting X converges to a positive number 
along any time trajectory almost surely. 

Stationary distributions of log-linear dynamics can be explicitly constructed for the class of 
the so-called potential games [18, 19]. In such games, if any single player changes his strategy, 
then the payoff differences are the same for all players. More precisely, a game is a potential 
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game if there exists a function p : S x S x S ^ R, invariant under any permutation of 
arguments such that ior all x,x',y,z & S 

U {x', y,z)-U {x, y, z) = p{x', y, z) - p{x, y, z) (4) 

We call this function a potential of the game. 

It is easy to see that p{A, A, A) = a, p{A, A, B) = 0, p{A, B, B) = b, p{B, B,B) = b + c is 
a potential of an elementary three-player game defined in (1). For players on the triangular 
lattice playing six elementary games, for any X e fl\, 

p{X)^ Y: p{X,,Xj,X,), (5) 

(iJ,fe)eAcA 

is a potential of the configuration X, where a sum is taken with respect to all elementary 
triangles in A. We have to stress here that even if an elementary game has a potential it does 
not necessarily mean that a resulting spatial game has a potential. This depends upon the 
spatial structure of interactions as we will see in Section 4. 

We will now show that the following probability distribution is the unique stationary state 
of our spatial game. 

Proposition 



is the stationary state of a three-player game on the triangular lattice. 
Proof: 

We will show that p^ satisfies the detailed balance condition 

pi{X)P{X,Y)^pi{Y)P{Y,X) (6) 

for all X, y G ^a, where P{X, Y) is the transition probability from X to Y given in (3). 
Then it follows that p^ is a stationary distribution because 

Y: p1{X)P{X,Y)= Y: l^i{Y)P{Y,X) = pi{Y) Y P{Y,X)=pi{Y). 

Let us observe that Y can be different at at most one lattice site, say i, which was chosen 
randomly (with probabihty 1/|A|) out of A. Let 
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We have 



= £)e^E(i,,-,fe)sA '=(^-^i'^fe)e/^Eo,fc) u{x,,Xj,Xk)-p{Xi,Xj,Xk)+p{Yi,Xj,Xk) 

so /^^ satisfies the detailed balance condition which completes the proof of the proposition. 

/i^ is a so-called finite-volume Gibbs state - a probability distribution describing the equi- 
librium behavior of systems of many interacting particles. In the following section, we will 
investigate the stochastic stability of Nash configurations for different payoff parameters of 
three-player games. 

3 Stochastic and ensemble stability 

Different Nash configurations of a given game usually have different values of a potential. It 
follows from the explicit form of the stationary state in the Proposition that Nash configurations 
with the maximal potential are stochastically stable. We obtain immediately the following 
theorems. 

Theorem 1 Let a,c > 0. If a > h + c, then is stochastically stable; if a < b + c, then X^ 
is stochastically stable. 

Let us notice that in our case, stochastically stable configurations appear in the long run 
with the probability 1 in the zero-noise limit. 

li a = b + c, then both X^ and X^ are stochastically stable and in the limit of zero noise 
they occur with the probability 1/2. 

Theorem 2 Let a > 0, c < 0, and b> 0. If a > b, then X^ is stochastically stable; if a <b, 
then X^^^ and its two translates are stochastically stable. 

li a — b, then all four Nash configurations are stochastically stable and they occur with the 
probability 1/4. 

Wc sec that if there are two or more Nash configurations with the maximal potential, then 
the problem of equilibrium selection is still not resolved. 
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Let us notice that limA_»z2 A*a(-'^) — for every X e S^, where L is the infinite triangular 
lattice. Hence for large A and any nonzero nisewe may only observe, with reasonable positive 
frequencies, ensembles of configurations and not particular configurations. It may happen that 
the stationary state is highly concentrated on an ensemble consisting of one Nash configuration 
and its small perturbations, i.e. configurations, where most players play the same strategy. We 
will call such configurations low-noise ensemble stable. 

Definition 3 X e Qa is low- noise ensemble stable if for every e > 0, there exists /3(e) such 
that for every /3 > /5(e) there exists A(/3) such that H\{Y G I^a; Yi ^ Xj) < e for any i G A if 
A(/5) C A. 

If X is low-noise ensemble stable, then the ensemble consisting of X and configurations which 

are different from X at few sites has probability close to one in the stationary distribution. 
It may happen that only one of many stochastically stable Nash configurations is low-noise 
ensemble stable. We will show that this is exactly the case of three-player games with certain 
payoff parameters. 

We will first consider the case of a, c > 0, & < 0, a = 6 -|- c and therefore a < c. 

We perform first the limit A — > and obtain a so-called infinite- volume Gibbs state 

/ = lim (7) 

We may then apply a technique developed by Bricmont and Slawny [20, 21]. They studied 
low-temperature stability of the so-called dominant ground-state configurations. It follows 
directly from Theorem A in [21] that 

ii\X, ^A)>\- e{P) (8) 

for any site i of the lattice and e{f3) ^ as /5 — oo. For 6 > so a > c we have the 
analogous inequalityfor the strategy B. The following theorem is a simple consequence of 
above inequalities. 

Theorem 3 Let a = b + c. If b < 0, then X^ is low-noise ensemble stable and if b > 0, then 
X^ is low-noise ensemble stable. 

Theorems 1 and 3 say that for any low but fixed level of noise and 6 < 0, if the number 
of players is big enough, then in the long run, almost all players use A strategy. On the other 
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hand, if for any fixed number of players, the noise level is lowered substantially, then both 
strategies appear with frequencies close to 1/2. 

Let us sketch briefly the reason of such a behavior. We assume that a < c. While it is 
true that both Nash configurations have the same potential which is one-third of the payoff 
of the whole system (it plays the role of the total energy of a system of interacting particles), 
the Nash configuration has more lowest-cost excitations. Namely, if one player changes 
his strategy to then the potential of the configuration decreases by 6a. If one player in the 
Nash configuration changes his strategy to A, the potential of the configuration decreases 
by 6c > 6a. Now, the probability of the occurrence of any configuration in the Gibbs state 
(which is the stationary distribution of our stochastic dynamics) depends on the total payoff in 
an exponential way. One then proves that the probability of an ensemble consisting of the X-^ 
Nash configuration and configurations which are different from it at few sites only is much bigger 
than the probability of the analogous X^-ensemble. On the other hand, configurations which 
are outside X-^ and X^-ensembles appear with exponentially small probabilities. It means 
that for large enough systems (and small but not extremely small noise level) we observe in the 
stationary distribution the X^ Nash configuration with perhaps few different strategics. The 
above argument was made into a rigorous proof for infinite systems of corresponding lattice-gas 
models of interacting particles by Bricmont and Slawny in [20, 21]. They would call X^ a 
dominant ground-state configuration. 

We have an analogous theorem for the other class of three-player games. 

Theorem 4 For a > 0, c < 0, and a — b, if a < \c\, then X^ is low-noise ensemble stable; if 
a > \c\, then X^^^ and its translates are low-noise ensemble stable. 

Here the lowest-cost excitation from X"^ is still 6a. Let us describe the lowest-cost excitations 
from X^^^. When B changes to A, then the payoff of the configuration decreases by 6fe = 6a. 
However, if A changes to B, the payoff decreases by |c|. Therefore, if a > |c|, then X^^^ has 
more lowest-cost excitations and hence is low-noise ensemble stable. 

4 Nonpotential three-player games 

Now we will consider an example of a three-player spatial game without a potential. Players 
are now placed on a finite subset of the one-dimensional regular lattice Z (for simplicity we 
will assume periodic boundary conditions and therefore agents will reside on a circle). Every 
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agent can play only one three-player game with his right and left nearest neighbor. Although 
any single game with a payoff matrix given in (1) has a potential as before but a sum of 
three-player interactions is not a potential of the spatial game. The reason for this is that if 
any agent chooses a best-response strategy, he does not take into account a game with two 
left or two right neighbors. However, his action may change their payoffs as a result of two 
additional three-player games. Hence, /i^ given in the Proposition is no longer a stationary 
state of our stochastic dynamics. To find stochastically stable configurations, we must resort 
to different methods. We will use the following tree representation of stationary states of 
irreducible Markov chains [22]. Let {fl,P) be an irreducible Markov chain with a finite state 
space Q and the transition probabihties given by P : Q x Q — > [0, 1]. Let us denote by its 
unique stationary distribution. For X e Q, let an X-tree be a directed graph on Q such that 
from every Y ^ X there is a unique path to X and there are no outcoming edges out of X. 
Denote by T{X) the set of all X-trees and let 

E n PiY,Z), (9) 
deT{x) {Y,z)ed 

where the product is with respect to all edges of d. The following representation of a stationary 
distribution n was provided by Preidhn and Wentzell [22] : 

for all X en. 

We will now use the above characterisation of a stationary distribution to find stochastically 
stable states in our nonpotential game for the case of a, c > 0. 

Let us note that X^ and X^ are the only absorbing states of the noise-free dynamics. 
When we start with any state different from X^ and X^, then after a finite number of steps 
of the best-response dynamics we arrive at either X"^ or X^ and then stay there forever. It 
follows from the above tree representation of the stationary distribution that any state different 
from X"^ and X^ has zero probability in the zero-noise limit. Moreover, in order to study the 
zero-noise limit of the stationary distribution, it is enough to consider paths between absorbing 
states. More precisely, we construct X-trees with absorbing states as vertices. The family of 
such trees is denoted hy T{X) Let 

{Y,Z)ed 
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where -P(Y, Z) — maxn(w^,vK') P(^: W): ^^^^ the last product is taken along any path joining 
Y with Z on the full graph and the maximum is taken with respect to all such paths. 

Now we may observe that in our three- player game, if limp^^qmlX^) /q^{X^) — 0, then 
is stochastically stable. Therefore we have to compare trees with biggest Qm in (11); such 
trees we call maximal. 

Now we will use the above tree representation of a stationary state in two different noise 
models. We begin with a stochastic dynamics with a state-independent noise. Namely, at each 
discrete moment of time, a randomly chosen agent plays the best response with the probability 
1 — e and with the probability e he makes a mistake. Below we assume that a, c > so there 
are two Nash configurations, X"^ and X^. 

Theorem 5 For the state-independent noise, ifb<0, then X^ is stochastically stable, ifb>0, 
then X^ is stochastically stable. 

Proof: The theorem follows from the observation that if 6 < 0, then qm{X^) is of order e and 
qm{X^) is of order e'^'/^, and if 6 > 0, then it is the other way around. 

Now we come back to the state-dependent log-linear noise. 

Theorem 6 For the log-linear noise, if a < c, then for every small b < 0, there is K{b) such 
that X^ is stochastically stable if |A| > K{b) and X^ is stochastically stable if |A| < K{b). 

Proof: If |6| < a, then we get 

^miX"") = (^^g;3a)(l + e-/36)|A|-2(l + g-/3c)- (l^) 

We also have that limfe^o K{b) =00. 

For a > c and 6 < 0, it follows from the above expressions of qm(X^) and qm{X^) that X^ 
is stochastically stable for any number of players. We see that in nonpotential games stochastic 
stability may depend upon the number of players. Let us notice that for any arbitrarily large 
c and 6 < 0, if the number of players is sufficiently big, then in the zero-noise limit, all of them 
play the inefficient strategy A which gives them the lower payoff than the strategy B in the 
configuration X^. 
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5 Summary 



To address the problem of equilibrium selection in spatial games with many players, we in- 
troduced the concept of low-noise ensemble stability. We showed that in certain symmetric 
three-player games with two strategies, there exist Nash configurations that are stochastically 
stable but not low-noise ensemble stable. It means that for any arbitrarily low but fixed noise, 
if the number of players is large enough, then some stochastically stable strategies are played 
with arbitrarily small frequencies. We also showed that for nonpotential three-player games, 
stochastic stability may depend upon the number of players. 
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